Sensory Integration and Kalman Filtering

For the sake of concreteness, let’s think about the problem of estimating visual depth. A commonly assumed framework for how an observer might go about judging the depth of a visual object defined by multiple visual cues is the following two-stage process. First, depth estimates based on individual cues are derived. Next, a weighted combination of these estimates is calculated and used as the observer’s composite depth percept; the cue weights are based on the relative reliabilities of the cues in the current visual context. For example, consider an observer judging the depth of an object defined by motion and texture cues. During stage one, the observer calculates depth estimates based on each individual cue. Let dM (m) denote the observer’s depth-from-motion estimate, and let dT (t) denote the observer’s depth-from-texture estimate. During stage two, the observer combines these estimates into a unified depth percept, denoted d(m, t), using, for instance, a linear cue combination rule: